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Summary of Review 


A recent report offers a how-to guide for reform advocates interested in removing commu- 
nities’ democratic control over their schools. The report explains how these reformers can 
influence states to use the Every Student Succeeds Act (ESSA) Title I school improvement 
funds to support a specific set of reforms: charter schools, state-initiated turnarounds, and 
appointment of an individual with plenipotentiary authority over districts or schools. While 
the report acknowledges that the research evidence on the effectiveness of these reforms as 
school improvement strategies is limited, it uses a few exceptional cases to explain how ad- 
vocates seeking to influence the development of state ESSA plans can advance them anyway. 
As this review explains, support for the effectiveness of these approaches is simply too limit- 
ed to present them as promising school improvement strategies. The report omits research 
that evaluates the models relative to other school reform initiatives, and it fails to take into 
account the opportunity costs of pursuing one set of policies over another. It also relies on 
test-score outcomes as the sole measure of success, thus ignoring other impacts these strat- 
egies may have on students and their local communities or the local school systems where 
they occur. For these reasons, policymakers, educators and state education administrators 
should be wary of relying on this report to guide them as they develop their state improve- 
ment plans and consider potential strategies for assisting low-performing schools and dis- 
tricts. 
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NEPC ReEvIEw: 
LEVERAGING ESSA TO SUPPORT QUALITY-SCHOOL GROWTH 


Gail L. Sunderman, University of Maryland 


|. Introduction 


The Thomas B. Fordham Institute and Education Cities released a report on March 30, 2017 
titled, Leveraging ESSA to Support Quality-School Growth.* 


Nelson Smith, a consultant on education policy and public charter schooling and former 
president and CEO of the National Alliance for Public Charter Schools, and Brandon Wright, 
editorial director for Fordham, authored the report. Education Cities consists of 31 organi- 
zations in 26 cities whose aim is to promote neo-liberal reform strategies. 


This report is a how-to guide designed to show how Education City members, other city- and 
state-based stakeholders, and education reform advocates can influence states to use the 
Every Student Succeeds Act (ESSA) Title I school improvement funds to support a specific 
set of reforms. It identifies three models or approaches to school improvement that it pro- 
motes: charter expansion, state turnaround districts, and state-led, district-based solutions. 
The report defines each of these as follows: 


e Charter expansion is an opportunity to convert or replace schools identified for 
comprehensive and targeted support with charters. 


e State turnaround districts occur when the state removes control of a school from 
the district, and creates a “state-led entity” to operate the school. 


¢ With a state-led, district-based solution, the state removes the powers of superin- 
tendents and school boards and vests that authority in a single individual. 


The report begins with a review of ESSA Title I school improvement provisions, arguing 
that ESSA “opens the door to a variety of strategies directly managed by the state and/or 
carried out through intermediaries chosen by the state...” (p. 6). By delegating to the states 
responsibility for identifying interventions for low performing schools, the report argues 
that ESSA offers opportunities for advocates to leverage the law to support charters or gov- 
ernance-based interventions (i.e., turnaround districts and receiverships). 


The report highlights the tightened evidentiary requirements that reform initiatives must 
meet for an intervention to be adopted by states and districts under ESSA, noting that “in 
the case of governance-based interventions (turnaround districts and receiverships, for ex- 
ample), the quandary is that the interventions themselves have not been subjected to the 
most rigorous forms of evaluation” (p. 7). To work around the lack of rigorous evaluation, 
the report makes the spurious claim that a particular model “. . . can be justified if there is 
a convincing argument that it will deliver school-level action that meets the [ESSA] tier 1-3 
standard” (p. 8). The report goes on to explain how advocates can influence states to adopt 
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charters or governance-based strategies as school improvement strategies. 


Scattered throughout, the report identified “action opportunities,” or recommendations for 
advocates to consider as they decide on their plan of action. For example, when seeking to 
influence the development of state ESSA plans, it “strongly recommends that members (i.e., 
of Education Cities) and their colleagues stress the importance of creating a competitive 
grant process ... Members can offer to help SEAs design the RFP” (p. 11). It also encourages 
Education Cities members to be cognizant of local political currents, noting that some of the 
reform models may not have local political support and others will require “hard bargain- 


ing” (p. 13). 


Il. Findings and Conclusions of the Report 


The purpose of this report is to prompt action among Education Cities members and others 
to take advantage of the changes in ESSA to influence states as they write their state ac- 
countability plans. In particular, it focuses on using the mandatory seven percent set aside 
in Title I. By outlining the ESSA requirements and evidentiary standards, it aims to equip 
members with rudimentary information to address challenges they may encounter when 
advocating for these models. The main conclusions of the report are as follows: 


1. State ESSA plans are in the works so Education Cities members and other educa- 
tion leaders should act soon to help design and implement these plans. 


2. There are three models they think worth considering as strategies that states 
might use the Title I school improvement set-aside: charter expansion, state 
turnaround districts, and state-led, district-based solutions. 


3. Of the three models, they claim that charter school expansion is most likely to 
meet the ESSA’s evidence requirements. 


4. There is little sophisticated evaluation of the state turnaround districts and the 
state-led, district-based strategies; nonetheless strategies that modify gover- 
nance arrangements can meet the tier 1-3 ESSA standards. 


5. ESSA, unlike NCLB, permits states to choose school improvement strategies as 
long as they meet the ESSA standards of evidence. 


Ill. The Report’s Rationale for Its Findings and Conclusions 


The report’s conclusions are interesting because, while they cite research showing the lim- 
itations of the suggested reform models, the report goes on to recommend ways that advo- 
cates and others can use the ESSA requirements to promote the adoption of them as school 
improvement strategies. The rationale it uses for advocating for any of the three models 
from an evidence perspective is that if there is at least one study that meets any of the ev- 
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idence-based standards, even the lowest standard, that is enough to consider a particular 
model. The report accompanies these conclusions with a number of caveats—“what works 
in one place may fail in another”—that are aimed at preparing advocates for any potential 
pushback or unfavorable outcomes they may encounter when choosing and promoting a 
particular model. 


IV. The Report’s Use of Research Literature 


The use of the research literature is selective. It includes no peer-reviewed journal articles 
and many of the cited references are themselves advocacy pieces. On charters, it draws pri- 
marily from a 2015 study of urban charter schools conducted by the Center for Research on 
Education Outcomes (CREDO) to show that charters positively impact student outcomes.? 


While CREDO has done significant work on the effectiveness of charter schools, the meth- 
odology used in the 2015 study contains a variety of technical and conceptual issues that 
raise questions about its findings. Because of these methodological concerns, Maul (2015b) 
concluded, “the findings from the CREDO studies cannot be regarded as compelling evi- 
dence of the greater effectiveness of charter schools compared to traditional public schools, 
either overall or specifically within urban districts.” Even setting aside the concerns over 
the analytic methods, “the actual effect sizes reported are very small, explaining well under 
a tenth of one percent of the variance in tests scores. To call such an effect “substantial” 
strains credulity.”5 


The report’s review of charters recognizes that the performance of charters varies consider- 
ably, from “amazing to awful” but then asserts, without citing evidence that “when overseen 
by vigilant authorizers, charters can achieve powerful results . . .” (p. 13). It also acknowl- 
edges that charters as a turnaround initiative “has been little studied” and that there are 
few charter operators that specialize in school turnarounds (p. 14). Nonetheless, it asserts 
without evidence that charter expansion makes sense as a turnaround strategy “if we’re 
focused on turning around students’ lives rather than school buildings” and targets urban 
areas “where success has been best documented” (p. 14). It is unclear what turning around 
school buildings means and, as previously noted, the evidence from the CREDO report on 
the success of charters in urban areas is not compelling. 


The report omits a rather large body of research on charter schools and market-oriented 
reforms that could provide a more nuanced review of the evidence on charter schools, and 
does not evaluate charters relative to other school reform initiatives that hold greater prom- 
ise at less cost.° 


The evidence on state turnaround districts draws on a few reports and newspaper articles 
in three states (Louisiana, Tennessee, and Nevada). The report cites two research studies 
that examined the School Recovery District (SRD) put in place in New Orleans following 
Hurricane Katrina.’ The report overstates the positive effects reported in the study® and 
fails to acknowledge the authors’ conclusion that the reforms put in place were multifaceted, 


http://nepc.colorado.edu/thinktank/review-ESSA-accountability 5 of 10 


that it is unclear whether the gains could be achieved in other cities without a tragedy like 
Hurricane Katrina, or that the analysis does not shed any light on which factors drove the 
improvements.? The report cites research that fails to find any positive impact on student 
outcomes in the turnaround districts in the other two states. Nonetheless, the report lays out 
action opportunities for those wanting to advocate state turnaround as a school improve- 
ment strategy. 


For the third model—state-led, district-based solutions—the report acknowledges that few 
states have tried this model. It relies on two examples, a state takeover of the Lawrence, 
Massachusetts school district and “a watered-down version in New York State” (p. 22). The 
evidence that the model is working in Lawrence, MA relies on one report issued by the 
Massachusetts Department of Elementary and Secondary Education that was prepared by 
the school district and the receiver put in charge of implementing the district’s turnaround 
plan.'’? While this report showed improvements in student outcomes, it is not a rigorous 
evaluation that controls for unobserved differences between students. Beyond listing the 
strategies put in place under the receivership, it does not clarify which aspects of the take- 
over are responsible for the observed impacts or whether the Lawrence program could be 
replicated in other districts." 


In sum, the evidence cited to support a claim of effectiveness is too limited to draw the 
conclusion that these three models are promising school improvement strategies. Few of 
the studies cited meet the criteria for identifying evidence recommended by the U.S. De- 
partment of Education non-regulatory guidance.’ While Congress has repealed these regu- 
lations, they have not been replaced. Thus, this is the only government document offering 
guidance on how to interpret the law’s evidence-based requirements. In addition, none of 
the models are evaluated relative to other school reform initiatives and thus fails to take 
into account the opportunity costs of pursuing one set of policies over another. Finally, the 
report relies on student outcomes as the sole measure of success. This ignores other impacts 
these strategies may have on students and their local communities or the local school sys- 
tems where they occur. 


V. Review of the Validity of the Findings and Conclusions 


This report is an advocacy piece using a few exceptional cases to provide a rationale that 
Education Cities members can use to influence states to use their Title I school improvement 
monies to support charter schools and school governance takeovers of schools and districts. 
Thus, evidence matters only in so far as it can be used to advance a particular policy prefer- 
ence. 


All three strategies lack a valid foundation in the literature. Charter schools do not perform 
better than traditional public schools, there is no evidence that state-initiated turnarounds 
have been successful, and the success of plenipotentiary approaches, where they have been 
tried, is weak. 
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VI. Usefulness of the Report for Guidance of Policy and Practice 


Policymakers, educators and state education administrators should be wary of relying on 
this report—or Education Cities members who use it—as a source of information that can 
guide them as they develop their state improvement plans. The report provides very little 
useful guidance for states as they consider potential strategies for assisting low-performing 
schools and districts to improve. It is an advocacy piece with little to no foundation in the re- 
search literature. Given that charter schools are more racially and economically segregated 
than traditional public schools, the lack of evidence that state-initiated district and school 
takeovers improve performance of low-performing schools, and the anti-democratic effects 
of receiverships, these models are unlikely to promote school improvement and could have 
potentially negative consequences. 
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